Entropy, dimension and the Elton-Pajor Theorem

The Vapnik-Chervonenkis dimension of a set K in R^n is the maximal dimension of the coordinate cube of a given size, which can be found in coordinate projections of K. We show that the VC dimension of a convex body governs its entropy. This has a number of consequences, including the optimal Elton's theorem and a uniform central limit theorem in the real valued case

Remarks on the geometry of coordinate projections in R^n

We study geometric properties of coordinate projections. Among other results, we show that if a body K in R^n has an "almost extremal" volume ratio, then it has a projection of proportional dimension which is close to the cube. We compare type 2 and infratype 2 constant of a Banach space. This follows from a comparison lemma for Rademacher and Gaussian averages. We also establish a sharp estimate on the shattering dimension of the convex hull of a class of functions in terms of the shattering dimension of the class itself.Comment: Israel Journal of Mathematics, to appea

Small ball probability for the condition number of random matrices

Let $A$ be an $n\times n$ random matrix with i.i.d. entries of zero mean, unit variance and a bounded subgaussian moment. We show that the condition number $s_{\max}(A)/s_{\min}(A)$ satisfies the small ball probability estimate $${\mathbb P}\big\{s_{\max}(A)/s_{\min}(A)\leq n/t\big\}\leq 2\exp(-c t^2),\quad t\geq 1,$$ where $c>0$ may only depend on the subgaussian moment. Although the estimate can be obtained as a combination of known results and techniques, it was not noticed in the literature before. As a key step of the proof, we apply estimates for the singular values of $A$, ${\mathbb P}\big\{s_{n-k+1}(A)\leq ck/\sqrt{n}\big\}\leq 2 \exp(-c k^2), \quad 1\leq k\leq n,$ obtained (under some additional assumptions) by Nguyen.Comment: Some changes according to the Referee's comment

Algorithmic linear dimension reduction in the l_1 norm for sparse vectors

This paper develops a new method for recovering m-sparse signals that is simultaneously uniform and quick. We present a reconstruction algorithm whose run time, O(m log^2(m) log^2(d)), is sublinear in the length d of the signal. The reconstruction error is within a logarithmic factor (in m) of the optimal m-term approximation error in l_1. In particular, the algorithm recovers m-sparse signals perfectly and noisy signals are recovered with polylogarithmic distortion. Our algorithm makes O(m log^2 (d)) measurements, which is within a logarithmic factor of optimal. We also present a small-space implementation of the algorithm. These sketching techniques and the corresponding reconstruction algorithms provide an algorithmic dimension reduction in the l_1 norm. In particular, vectors of support m in dimension d can be linearly embedded into O(m log^2 d) dimensions with polylogarithmic distortion. We can reconstruct a vector from its low-dimensional sketch in time O(m log^2(m) log^2(d)). Furthermore, this reconstruction is stable and robust under small perturbations

Dimension-adaptive bounds on compressive FLD Classification

Efficient dimensionality reduction by random projections (RP) gains popularity, hence the learning guarantees achievable in RP spaces are of great interest. In finite dimensional setting, it has been shown for the compressive Fisher Linear Discriminant (FLD) classifier that forgood generalisation the required target dimension grows only as the log of the number of classes and is not adversely affected by the number of projected data points. However these bounds depend on the dimensionality d of the original data space. In this paper we give further guarantees that remove d from the bounds under certain conditions of regularity on the data density structure. In particular, if the data density does not fill the ambient space then the error of compressive FLD is independent of the ambient dimension and depends only on a notion of ‘intrinsic dimension'

Greedy Signal Recovery Review

The two major approaches to sparse recovery are L1-minimization and greedy methods. Recently, Needell and Vershynin developed Regularized Orthogonal Matching Pursuit (ROMP) that has bridged the gap between these two approaches. ROMP is the first stable greedy algorithm providing uniform guarantees. Even more recently, Needell and Tropp developed the stable greedy algorithm Compressive Sampling Matching Pursuit (CoSaMP). CoSaMP provides uniform guarantees and improves upon the stability bounds and RIC requirements of ROMP. CoSaMP offers rigorous bounds on computational cost and storage. In many cases, the running time is just O(NlogN), where N is the ambient dimension of the signal. This review summarizes these major advances

Eigenvalue variance bounds for Wigner and covariance random matrices

This work is concerned with finite range bounds on the variance of individual eigenvalues of Wigner random matrices, in the bulk and at the edge of the spectrum, as well as for some intermediate eigenvalues. Relying on the GUE example, which needs to be investigated first, the main bounds are extended to families of Hermitian Wigner matrices by means of the Tao and Vu Four Moment Theorem and recent localization results by Erd\"os, Yau and Yin. The case of real Wigner matrices is obtained from interlacing formulas. As an application, bounds on the expected 2-Wasserstein distance between the empirical spectral measure and the semicircle law are derived. Similar results are available for random covariance matrices